THE BATTLE OF NEIGHBORHOODS¶
New York City's demographics show that it is a large and ethnically diverse metropolis. With it's diverse culture , comes diverse food items. There are many resturants in New york City, each beloning to different categories like Chinese , Indian , French etc.
So as part of this project , we will list and visualize all major parts of New York City that has great indian resturants.
Methodology:¶
- We begin by collecting the New York city data from "https://cocl.us/new_york_dataset".
- We will find all venues for each neighborhood using FourSquare API.
- We will then filter out all Indian Restuarant venues.
- Next using FourSquare API, we will find the Ratings, Tips, and Like count for all the Indian Resturants.
- Next we will sort the data keeping Ratings as the constraint.
- Finally, we will visualize the Ranking of neighborhoods using python's Folium library.
In [78]:
# required libraries
import pandas as pd
import numpy as np
pd.set_option('display.max_columns', None)
pd.set_option('display.max_rows', None)
import requests
from bs4 import BeautifulSoup
#!pip install geocoder
import geocoder
import os
!pip install folium
import folium # map rendering library
from geopy.geocoders import Nominatim # convert an address into latitude and longitude values
# Matplotlib and associated plotting modules
import matplotlib.pyplot as plt
import matplotlib.cm as cm
import matplotlib.colors as colors
%matplotlib inline
In [79]:
# function to get the geocodes i.e latitude and longitude of a given location
def geo_location(address):
# get geo location of address
geolocator = Nominatim(user_agent="ny_explorer")
location = geolocator.geocode(address)
latitude = location.latitude
longitude = location.longitude
return latitude,longitude
In [184]:
# function to intract with FourSquare API and will return the venue id , venue name and category.
def get_venues(lat,lng):
#set variables
radius=1000
LIMIT=100
CLIENT_ID = '#'# Foursquare ID, note there is a daily call quota limit
CLIENT_SECRET ='#' # Foursquare Secret, note there is a daily call quota it
VERSION = '20201118' # Foursquare API version
#url to fetch data from foursquare api
url = 'https://api.foursquare.com/v2/venues/explore?&client_id={}&client_secret={}&v={}&ll={},{}&radius={}&limit={}'.format(
CLIENT_ID,
CLIENT_SECRET,
VERSION,
lat,
lng,
radius,
LIMIT)
# get all the data
results = requests.get(url).json()
venue_data=results["response"]['groups'][0]['items']
venue_details=[]
for row in venue_data:
try:
venue_id=row['venue']['id']
venue_name=row['venue']['name']
venue_category=row['venue']['categories'][0]['name']
venue_details.append([venue_id,venue_name,venue_category])
except KeyError:
pass
column_names=['ID','Name','Category']
df = pd.DataFrame(venue_details,columns=column_names)
return df
In [193]:
# function to get venue details like like count , rating , tip counts for a given venue id.
def get_venue_details(venue_id):
CLIENT_ID = '#'# Foursquare ID, note there is a daily call quota limit
CLIENT_SECRET ='#' # Foursquare Secret, note there is a daily call quota it it
VERSION = '20201118' # Foursquare API version
#url to fetch data from foursquare api
url = 'https://api.foursquare.com/v2/venues/{}?&client_id={}&client_secret={}&v={}'.format(
venue_id,
CLIENT_ID,
CLIENT_SECRET,
VERSION)
# get all the data
results = requests.get(url).json()
venue_data=results['response']['venue']
venue_details=[]
try:
venue_id=venue_data['id']
venue_name=venue_data['name']
venue_likes=venue_data['likes']['count']
venue_rating=venue_data['rating']
venue_tips=venue_data['tips']['count']
venue_details.append([venue_id,venue_name,venue_likes,venue_rating,venue_tips])
except KeyError:
pass
column_names=['ID','Name','Likes','Rating','Tips']
df = pd.DataFrame(venue_details,columns=column_names)
return df
In [82]:
# Funtion to get the new york city data such as Boroughs, Neighborhoods along with their latitude and longitude.
def get_new_york_data():
url='https://cocl.us/new_york_dataset'
resp=requests.get(url).json()
# all data is present in features label
features=resp['features']
# define the dataframe columns
column_names = ['Borough', 'Neighborhood', 'Latitude', 'Longitude']
# instantiate the dataframe
new_york_data = pd.DataFrame(columns=column_names)
for data in features:
borough = data['properties']['borough']
neighborhood_name = data['properties']['name']
neighborhood_latlon = data['geometry']['coordinates']
neighborhood_lat = neighborhood_latlon[1]
neighborhood_lon = neighborhood_latlon[0]
new_york_data = new_york_data.append({'Borough': borough,
'Neighborhood': neighborhood_name,
'Latitude': neighborhood_lat,
'Longitude': neighborhood_lon}, ignore_index=True)
return new_york_data
In [83]:
new_york_data=get_new_york_data()
new_york_data.head()
new_york_data.shape
Out[83]:
The above result shows that there are 306 different Neighborhoods in New York.
In [84]:
# create a BAR PLOT to show different Neighborhoods in New York
plt.figure(figsize=(9,5), dpi = 100)
# title
plt.title('Number of Neighborhood for each Borough in New York City')
#On x-axis
plt.xlabel('Borough', fontsize = 15)
#On y-axis
plt.ylabel('No.of Neighborhood', fontsize=15)
#giving a bar plot
new_york_data.groupby('Borough')['Neighborhood'].count().plot(kind='bar')
#legend
plt.legend()
#displays the plot
plt.show()
From the above Bar Plot, we can see that Queens has highest number of neighborhoods.
In [85]:
# prepare neighborhood list that contains indian resturants
column_names=['Borough', 'Neighborhood', 'ID','Name']
indian_rest_ny=pd.DataFrame(columns=column_names)
count=1
for row in new_york_data.values.tolist():
Borough, Neighborhood, Latitude, Longitude=row
venues = get_venues(Latitude,Longitude)
indian_resturants=venues[venues['Category']=='Indian Restaurant']
print('(',count,'/',len(new_york_data),')','Indian Resturants in '+Neighborhood+', '+Borough+':'+str(len(indian_resturants)))
for resturant_detail in indian_resturants.values.tolist():
id, name , category=resturant_detail
indian_rest_ny = indian_rest_ny.append({'Borough': Borough,
'Neighborhood': Neighborhood,
'ID': id,
'Name' : name
}, ignore_index=True)
count+=1
In [86]:
indian_rest_ny.head()
indian_rest_ny.shape
Out[86]:
From the above result, we see that there are 143 Indian Resturants across New York City.
In [87]:
# BAR PLOT to show Number of Indian Resturants for each Borough in New York City.
plt.figure(figsize=(9,5), dpi = 100)
# title
plt.title('Number of Indian Resturants for each Borough in New York City')
#On x-axis
plt.xlabel('Borough', fontsize = 15)
#On y-axis
plt.ylabel('No.of Indian Resturants', fontsize=15)
#giving a bar plot
indian_rest_ny.groupby('Borough')['ID'].count().plot(kind='bar')
#legend
plt.legend()
#displays the plot
plt.show()
From the above Bar Plot, we can see that Queens has highest number of Indian resturants.
In [88]:
plt.figure(figsize=(9,5), dpi = 100)
# title
plt.title('Number of Indian Resturants for each Neighborhood in New York City')
#On x-axis
plt.xlabel('Neighborhood', fontsize = 15)
#On y-axis
plt.ylabel('No.of Indian Resturants', fontsize=15)
#giving a bar plot
indian_rest_ny.groupby('Neighborhood')['ID'].count().nlargest(5).plot(kind='bar')
#legend
plt.legend()
#displays the plot
plt.show()
In [187]:
indian_rest_ny[indian_rest_ny['Neighborhood']=='Floral Park']
Out[187]:
We can see that, Floral Park in Queens has the highest number of Indian Resturants with a total count of 8
In [ ]:
# get the prepare neighborhood list that contains indian resturants
column_names=['Borough', 'Neighborhood', 'ID','Name','Likes','Rating','Tips']
indian_rest_stats_ny=pd.DataFrame(columns=column_names)
#print(indian_rest_ny.values[:50])
count=1
for row in indian_rest_ny.values.tolist():
Borough,Neighborhood,ID,Name=row
try:
venue_details=get_venue_details(ID)
print(venue_details)
id,name,likes,rating,tips=venue_details.values.tolist()[0]
except IndexError:
print('No data available for id=',ID)
# we will assign 0 value for these resturants as they may have been
#recently opened or details does not exist in FourSquare Database
id,name,likes,rating,tips=[0]*5
print('(',count,'/',len(indian_rest_ny),')','processed')
indian_rest_stats_ny = indian_rest_stats_ny.append({'Borough': Borough,
'Neighborhood': Neighborhood,
'ID': id,
'Name' : name,
'Likes' : likes,
'Rating' : rating,
'Tips' : tips
}, ignore_index=True)
count+=1 #ranking of each resturant for further analysis.
In [212]:
indian_rest_stats_ny.head()
Out[212]:
In [214]:
indian_rest_ny.shape
Out[214]:
In [196]:
# saving to csv
indian_rest_stats_ny.to_csv('indian_rest_stats_ny.csv', index=False)
In [197]:
# loading data from csv
indian_rest_stats_ny_csv=pd.read_csv('indian_rest_stats_ny.csv')
indian_rest_stats_ny_csv.shape
indian_rest_stats_ny_csv.head()
Out[197]:
In [198]:
indian_rest_stats_ny['Likes']=indian_rest_stats_ny['Likes'].astype('float64')
indian_rest_stats_ny['Tips']=indian_rest_stats_ny['Tips'].astype('float64')
Indian restuarants with Maximum Likes, Maximum Ratings and Maximum Tips.
In [199]:
# Resturant with maximum Likes
indian_rest_stats_ny.iloc[indian_rest_stats_ny['Likes'].idxmax()]
Out[199]:
In [200]:
# Resturant with maximum Rating
indian_rest_stats_ny.iloc[indian_rest_stats_ny['Rating'].idxmax()]
Out[200]:
In [201]:
# Resturant with maximum Tips
indian_rest_stats_ny.iloc[indian_rest_stats_ny['Tips'].idxmax()]
Out[201]:
visualize neighborhood with maximum average rating of resturants.
In [202]:
ny_neighborhood_stats=indian_rest_stats_ny.groupby('Neighborhood',as_index=False).mean()[['Neighborhood','Rating']]
ny_neighborhood_stats.columns=['Neighborhood','Average Rating']
In [203]:
ny_neighborhood_stats.sort_values(['Average Rating'],ascending=False).head(10)
Out[203]:
Above are the top neighborhoods with top average rating of Indian resturants.
we will find the average rating of Indian Resturants for each Borough.
In [215]:
ny_borough_stats=indian_rest_stats_ny.groupby('Borough',as_index=False).mean()[['Borough','Rating']]
ny_borough_stats.columns=['Borough','Average Rating']
ny_borough_stats.sort_values(['Average Rating'],ascending=False).head()
Out[215]:
In [205]:
# visualization
plt.figure(figsize=(9,5), dpi = 100)
# title
plt.title('Average rating of Indian Resturants for each Borough')
#On x-axis
plt.xlabel('Borough', fontsize = 15)
#On y-axis
plt.ylabel('Average Rating', fontsize=15)
#giving a bar plot
indian_rest_stats_ny.groupby('Borough').mean()['Rating'].plot(kind='bar')
#legend
plt.legend()
#displays the plot
plt.show()
consider all the neighborhoods with average rating greater or equal 9.0 to visualize on map.m
In [206]:
ny_neighborhood_stats=ny_neighborhood_stats[ny_neighborhood_stats['Average Rating']>=9.0]
ny_neighborhood_stats
Out[206]:
In [217]:
# join this dataset to original new york data to get lonitude and latitude.
ny_neighborhood_stats=pd.merge(ny_neighborhood_stats,new_york_data, on='Neighborhood')
ny_neighborhood_stats=ny_neighborhood_stats[['Borough','Neighborhood','Latitude','Longitude','Average Rating']]
ny_neighborhood_stats
Out[217]:
In [208]:
# visualization on Map
ny_map = folium.Map(location=geo_location('New York'), zoom_start=12)
# instantiate a feature group for the incidents in the dataframe
incidents = folium.map.FeatureGroup()
# loop through the neighborhood and add each to the feature group
for lat, lng, in ny_neighborhood_stats[['Latitude','Longitude']].values:
incidents.add_child(
folium.CircleMarker(
[lat, lng],
radius=10, # define how big you want the circle markers to be
color='yellow',
fill=True,
fill_color='blue',
fill_opacity=0.6
)
)
In [220]:
ny_neighborhood_stats['Label']=ny_neighborhood_stats['Neighborhood']+', '+ny_neighborhood_stats['Borough']+'('+ny_neighborhood_stats['Average Rating'].map(str)+')'
In [221]:
# add pop-up text to each marker on the map
for lat, lng, label in ny_neighborhood_stats[['Latitude','Longitude','Label']].values:
folium.Marker([lat, lng], popup=label).add_to(ny_map)
# add incidents to map
ny_map.add_child(incidents)
Out[221]:
In [211]:
# visualization of Borough
ny_map = folium.Map(location=geo_location('New York'), zoom_start=12)
ny_geo = r'Borough_Boundaries.geojson'
map = ny_map.choropleth(
geo_data=ny_geo,
data=ny_borough_stats,
columns=['Borough', 'Average Rating'],
key_on='feature.properties.boro_name',
fill_color='YlOrRd',
fill_opacity=0.7,
line_opacity=0.2,
legend_name='Average Rating'
)
# display map
# as this is huge map data , we will save it to a file
ny_map.save('borough_rating.html')
Conclusion:¶
From our anaysis:
- Greenwich Village (Manhattan), Tribeca(Manhattan), West Village(Manhattan) are some of the best neighborhoods for Indian cuisine.
- Manhattan have potential Indian Resturant Market.
- Bronx ranks last in average rating of Indian Resturants.
- Manhattan is the best place to stay if you prefer Indian Cuisine.
In [ ]: